1. Field of the Invention
The present invention relates to computation, and, more particularly, to neural network information and signal processing.
2. Description of the Related Art.
Avionics sensor systems typically are confronted with the problem of identifying the emitters of various electromagnetic signals (i.e., radar pulses) being received. The known sensor systems typically include an antenna, a preprocessor, a pulse buffer, a digital signal processor, and a classifier. The antenna receives signals from various emitters with various frequencies, pulse widths, pulse repetition rates, locations, and so forth. The antenna output is preprocessed to extract a set of features for each received pulse. The set of features is processed by standard signal processing methods to cluster the pulses from each perceived emitter, and the classifier compares the features of each perceived emitter with pre-stored data to determine its identity. All of this analysis would take place in a homing missle which would then decide which of the perceived emitters to attack. However, the known sensor systems have the problem of extracting the emitter identity information from the mass of incoming pulses; the computing power required for standard digital signal processing in real time cannot be effectively put into every homing missle.
Attempts to understand the functioning of the human brain have led to various "neural network" models in which large numbers of neurons are interconnected. These models roughly presume each neuron exists in one of two states (quiescent and firing) with the state determined by the states of connected nuerons (if enough connected neurons are firing, then the original neuron should be in the firing state); and the thrust of the models is to perform computations such as pattern recognition with the neural networks. The models may be simple feedforward layered structures with an input layer of neurons, one or more hidden layers of neurons, and an output layer of neurons. Other models have feedback among the neurons and correspondingly more involved behavior.
J. Hopfield, Neural Networks and Physical Systems with Emergent Collective Computational Abilities, 79 Proc. Natl. Acad. Sci. USA 2554 (1982) describes a neural network model with N neurons each of which has the value 0 or 1 (corresponding to quiescent and to firing), so the state of the network is then a N-component vector V=[V.sub.1, V.sub.2, . . . , V.sub.N ]of O's and 1's which depends upon time. The neuron interconnections are described by a matrix T.sub.i,j defining the influence of the j.sup.th neuron on the i.sup.th neuron. The state of the network evolves in time as follows: each neuron i has a fixed threshold .theta..sub.i and readjusts its state V.sub.i randomly in time by setting V.sub.i equal to 0 or 1 depending on whether ##EQU1## is negative or positive. All neurons have the same average rate of readjustment, and the readjustments define a dynamic flow in state space.
With the assumption that T.sub.i,j is symmetric, the potential function ##EQU2## can be used to show that the flow of the network is to local minima of the potential function. Further, with a given set of uncorrelated N-component vectors U.sup.1, U.sup.2, . . . , U.sup.8, a T.sub.i,j can be defined by ##EQU3## and with the thresholds equal to 0, these U.sup.k are the fixed points of the flow and thus stable states of the network. This is a type of "outer product storage" of the vectors U.sup.1, U.sup.2, . . . , U.sup.8. Such a network can act as a content-addressable memory as follows: the memories to be stored in the network are used to construct the U.sup.k and hence T.sub.i,j, so the stored memories are fixed points of the flow. Then a given partial memory is input by using it to define the initial state of the network, and the state will flow usually to the closest fixed point/stable state U.sup.k which is then the memory recalled upon input of the partial memory.
Further analysis and modified network models appear in, for example, J. Hopfield et al, Computing with Neural Circuits: A Model, 233 Science 625 (1986) and J. Hopfield, Neurons with Graded Response Have Collective Computational Properties like Those of Two-State Neurons, 81 Proc. Natl. Acad. Sci. USA 3088 (1984).
L. Cooper, A Possible Organization of Animal Memory and Learning, Proc. Nobel Symp. Coll. Prop. Phys. Sys. 252 (Academic, New York 1973) observes that the modelling of neural network for animal memory and learning has the problem of mapping events in the animal's environment (i.e., sensory output) to signal distributions in the animal's neurons with the fundamental property of preserving closeness or separateness (in some sense not yet completely defined) of the events. That is, with a vector representation of the neural network states, two events as similar as a white cat and a gray cat should map into vectors which are close to parallel while two events as different as the sound of a bell and the sight of food should map into vectors that are close to orthogonal. Note that standard analysis, such as described in Gonzalez and Wintz, Digital Image Processing (Addison-Wesley 1977), does not use neural network computation and does not have this problem; rather, the standard analysis attempts to extract features and categorize by serial number crunching.
J. Anderson, Cognitive Capabilities of a Parallel System, NATO Advanced Research Workshop (Mar. 3, 1985) describes the Brain State in a Box (BSB) neural network model which includes outerproduct storage, Widrow-Hoff learning, and a ramped-threshold recall algorithm. That is, the matrix of interconnection strengths, T.sub.i,j, is modified to learn a new vector V.sub.j by ##EQU4## where .eta. is a learning constant and N is number of neurons. The learning constant is roughly the inverse of the number of times the matrix must be trained on a given vector before it fully learns the vector. The smaller the learning constant, the finer the resolution of the average direction for a learned state but the more times the input vectors must be trained. The learning procedure saturates when .DELTA.T.sub.i,j is close to zero, which implies that the vector is close to being an eigenvector of the matrix with an eigenvalue near 1.
Recall of a learned (stored) vector given an input vector U.sub.j is by the following interative process that converges towards an eigenvector: ##EQU5## where the "Limit" function clamps the values in the range from -1 to 1. The constants .gamma. and .beta. measure the feedback and signal decay in the algorithm. This synchronous recall algorithm replaces the dynamic flow of the Hopfield model. The usual applications of the BSB neural network such as data bases with words and letters encoded as their ASCII representations require binary neurons as in the first Hopfield model.